This document provides a summary of the change-point analysis for Dengue (dengue).

Piecewise CP Approach

The following section summarizes change-point analysis for trends in the number of visits before diagnosis using the standard piecewise-modeling approach to find the change-point. Specifically, we evaluate 4 peicewise models with linear, quadratic, cubic and exponential trends. The change-point is found by iterating over different change-points and selecting the best fitting model based on AIC.

SSD Visits

This section summarizes results using counts of SSD visits.

Change-point summary

The following table summarizes the optimal change-point for each trend type, with or without periodicity. For each approach, the change-point, prediction-bound change-point, and implied number of missed opportunities is reported.
label Change Point Pred. Bound CP CP # Miss PB CP # Miss
Piecewise lm w/ periodicity 7 18 1966.45 2452.60
Piecewise lm 7 18 1964.66 2444.54
Piecewise quad w/ periodicity 14 18 2214.20 2300.31
Piecewise quad 14 16 2208.73 2285.39
Piecewise cubic w/ periodicity 14 16 1935.21 1985.90
Piecewise cubic 21 16 2199.16 2173.21
Piecewise exp w/ periodicity 14 18 2045.31 2135.55
Piecewise exp 14 16 2038.02 2115.18

Performance Summary

The following table summarizes performance metrics for each model.
label Change Point Pred. Bound CP CP Consisency MSE MSE 7-day MSE 14-day
Piecewise lm w/ periodicity 7 18 -157.14286 276.79 4820.68 2813.04
Piecewise lm 7 18 -157.14286 286.42 5038.98 2998.57
Piecewise quad w/ periodicity 14 18 -28.57143 67.94 603.23 310.85
Piecewise quad 14 16 -14.28571 77.60 746.17 400.24
Piecewise cubic w/ periodicity 14 16 -14.28571 45.83 315.66 170.16
Piecewise cubic 21 16 23.80952 42.35 49.13 79.89
Piecewise exp w/ periodicity 14 18 -28.57143 71.27 647.28 342.43
Piecewise exp 14 16 -14.28571 81.86 745.93 404.87
The following table ranks each model in terms of the above performance metrics.
label Change Point Pred. Bound CP Average Rank consistency_rank mse_rank mse7_rank mse14_rank
Piecewise cubic 21 16 1.25 2 1 1 1
Piecewise cubic w/ periodicity 14 16 1.75 1 2 2 2
Piecewise quad w/ periodicity 14 18 3.00 3 3 3 3
Piecewise exp w/ periodicity 14 18 3.75 3 4 4 4
Piecewise quad 14 16 4.25 1 5 6 5
Piecewise exp 14 16 4.50 1 6 5 6
Piecewise lm w/ periodicity 7 18 6.25 4 7 7 7
Piecewise lm 7 18 7.00 4 8 8 8

Plot of each model that included periodicity

Plot of each model that did not include periodicity

All Visits

This section summarizes results using counts of all visits.

Change-point summary

The following table summarizes the optimal change-point for each trend type, with or without periodicity. For each approach, the change-point, prediction-bound change-point, and implied number of missed opportunities is reported.
label Change Point Pred. Bound CP CP # Miss PB CP # Miss
Piecewise lm w/ periodicity 7 16 2262.02 2692.39
Piecewise lm 7 16 2253.96 2667.72
Piecewise quad w/ periodicity 14 16 2548.36 2585.93
Piecewise quad 14 16 2529.15 2558.58
Piecewise cubic w/ periodicity 14 16 2217.03 2230.57
Piecewise cubic 14 9 2157.79 2038.04
Piecewise exp w/ periodicity 14 16 2311.97 2348.18
Piecewise exp 14 16 2269.79 2293.44

Performance Summary

The following table summarizes performance metrics for each model.
label Change Point Pred. Bound CP CP Consisency MSE MSE 7-day MSE 14-day
Piecewise lm w/ periodicity 7 16 -128.57143 428.06 5709.82 3230.02
Piecewise lm 7 16 -128.57143 1013.28 7904.00 5058.27
Piecewise quad w/ periodicity 14 16 -14.28571 223.22 657.65 450.33
Piecewise quad 14 16 -14.28571 805.89 2065.18 1706.23
Piecewise cubic w/ periodicity 14 16 -14.28571 193.28 335.52 298.62
Piecewise cubic 14 9 35.71429 778.97 1668.22 1493.67
Piecewise exp w/ periodicity 14 16 -14.28571 214.76 607.93 432.73
Piecewise exp 14 16 -14.28571 797.80 1931.53 1626.87
The following table ranks each model in terms of the above performance metrics.
label Change Point Pred. Bound CP Average Rank consistency_rank mse_rank mse7_rank mse14_rank
Piecewise cubic w/ periodicity 14 16 1.00 1 1 1 1
Piecewise exp w/ periodicity 14 16 1.75 1 2 2 2
Piecewise quad w/ periodicity 14 16 2.50 1 3 3 3
Piecewise cubic 14 9 3.75 2 5 4 4
Piecewise exp 14 16 4.25 1 6 5 5
Piecewise quad 14 16 5.00 1 7 6 6
Piecewise lm w/ periodicity 7 16 5.25 3 4 7 7
Piecewise lm 7 16 6.75 3 8 8 8

Plot of each model that included periodicity

Plot of each model that did not include periodicity

Bootstrapping CP Approach

SSD Visits

This section summarizes results using counts of SSD-related visits.

The following figure depicts the in-sample and out-of-sample performance (MSE) of various bounds on the opportunity window and different trends.

The following table depicts the top 10 specifications based on either aggregate or k-fold out-of-sample performance:

Aggregate Out-of-Sample
K-Fold Out-of-Sample
rank Bound (Days) Model MSE Bound (Days) Model MSE
1 14 Cubic 64.17 14 Cubic 116.61
2 21 Cubic 65.19 21 Cubic 118.62
3 21 Quadratic 71.29 21 Quadratic 125.26
4 14 Quadratic 82.35 14 Quadratic 134.84
5 7 Cubic 87.36 7 Cubic 138.91
6 21 Linear 90.08 21 Linear 145.03
7 28 Cubic 103.50 28 Cubic 157.84
8 28 Quadratic 105.53 28 Quadratic 160.12
9 28 Linear 113.33 28 Linear 168.30
10 14 Linear 128.36 14 Linear 181.44

The following figure depicts the observed and expected trend for the top 4 models based on aggregate out-of-sample performance:

The following figure depicts the observed and expected trend for the top 4 models based on 99-fold out-of-sample performance:

The following table depicts the 10 best models for each trend, based on aggregate out-of-sample performance:

Linear
Quadratic
Cubic
Rank Bound MSE Bound MSE Bound MSE
1 21 90.08 21 71.29 14 64.17
2 28 113.33 14 82.35 21 65.19
3 14 128.36 28 105.53 7 87.36
4 35 178.93 7 148.22 28 103.50
5 42 255.55 35 175.42 35 174.80
6 7 278.60 42 254.01 42 253.85
7 49 346.96 49 346.20 49 346.06
8 56 452.28 56 451.94 56 451.28
9 63 539.72 63 539.61 63 537.85
10 70 626.87 70 627.38 70 626.73

The following table depicts the 10 best models for each trend, based on 99-fold out-of-sample performance:

Linear
Quadratic
Cubic
Rank Bound MSE Bound MSE Bound MSE
1 21 145.03 21 125.26 14 116.61
2 28 168.30 14 134.84 21 118.62
3 14 181.44 28 160.12 7 138.91
4 35 234.75 7 199.65 28 157.84
5 42 311.75 35 230.96 35 230.08
6 7 330.85 42 310.18 42 309.92
7 49 403.77 49 403.05 49 402.86
8 56 509.84 56 509.62 56 509.01
9 63 598.07 63 598.10 63 596.51
10 70 686.45 70 686.98 70 686.39

Linear Models

The following figure depicts the top 4 performing linear models based on aggregate out-of-sample MSE:

The following figure depicts the top 4 performing linear models based on 99-fold out-of-sample MSE:

Quadratic Models

The following figure depicts the top 4 performing quadratic models based on aggregate out-of-sample MSE:

The following figure depicts the top 4 performing quadratic models based on 99-fold out-of-sample MSE:

Cubic Models

The following figure depicts the top 4 performing cubic models based on aggregate out-of-sample MSE:

The following figure depicts the top 4 performing cubic models based on 99-fold out-of-sample MSE:

All Visits

This section summarizes results using counts of all visits.

The following figure depicts the in-sample and out-of-sample performance of various bounds on the opportunity window and different trends.

The following table depicts the top 10 specifications based on both aggregate and k-fold out-of-sample performance:

Aggregate Out-of-Sample
K-Fold Out-of-Sample
rank Bound (Days) Model MSE Bound (Days) Model MSE
1 14 Cubic 245.82 14 Cubic 460.33
2 21 Cubic 263.96 21 Cubic 479.69
3 14 Quadratic 268.61 14 Quadratic 481.77
4 7 Cubic 270.26 7 Cubic 484.25
5 21 Quadratic 274.50 21 Quadratic 489.58
6 21 Linear 285.11 21 Linear 498.92
7 14 Linear 301.38 14 Linear 511.66
8 28 Cubic 326.82 28 Cubic 544.89
9 28 Quadratic 333.87 28 Quadratic 551.43
10 28 Linear 336.69 28 Linear 553.55

The following figure depicts the observed and expected trend for the top 4 models based on aggregate out-of-sample performance:

The following figure depicts the observed and expected trend for the top 4 models based on k-fold out-of-sample performance:

The following table depicts the 10 best models for each trend, based on aggregate out-of-sample performance:

Linear
Quadratic
Cubic
Rank Bound MSE Bound MSE Bound MSE
1 21 285.11 14 268.61 14 245.82
2 14 301.38 21 274.50 21 263.96
3 28 336.69 28 333.87 7 270.26
4 35 419.88 7 344.01 28 326.82
5 7 457.67 35 421.30 35 418.23
6 42 525.85 42 528.22 42 529.45
7 49 658.32 49 660.70 49 660.03
8 56 812.12 56 814.68 56 813.87
9 63 948.60 63 950.59 63 950.15
10 70 1087.20 70 1086.70 70 1088.31

The following table depicts the 10 best models for each trend, based on 99-fold out-of-sample performance:

Linear
Quadratic
Cubic
Rank Bound MSE Bound MSE Bound MSE
1 21 498.92 14 481.77 14 460.33
2 14 511.66 21 489.58 21 479.69
3 28 553.55 28 551.43 7 484.25
4 35 638.22 7 556.84 28 544.89
5 7 665.62 35 639.91 35 637.14
6 42 744.27 42 746.61 42 747.99
7 49 875.85 49 878.29 49 877.91
8 56 1027.95 56 1030.54 56 1030.03
9 63 1163.36 63 1165.32 63 1165.27
10 70 1301.27 70 1300.55 70 1302.09

Linear Models

The following figure depicts the top 4 performing linear models based on out-of-sample MSE

Quadratic Models

The following figure depicts the top 4 performing quadratic models based on out-of-sample MSE

Cubic Models

The following figure depicts the top 4 performing cubic models based on out-of-sample MSE